SRI  International 


6  May  2005 

Integrating  mission,  robot  localization  and  communication 
requirements  through  collaboration 

SRI  Project  No.:  PI 3648 

Contract/Grant  No.:  N00014-03-C-0264 

Report  Type:  A002  -  Final  Report 

Prepared  by: 

Dr.  Charles  L.  Ortiz,  Program  Manager 
Artificial  Intelligence  Center 

Prepared  for: 

Dr.  Clifford  W.  Anderson 
Deputy  Director  for  Discovery  and  Invention 
Expeditionary  Warfare  Operations  Technology  Division 
Code  353 

Office  of  Naval  Research 
800  North  Quincy  Street 
Arlington,  VA  22217-5660 


distribution  statement  a 

Approved  for  Public  Release 
'  Distribution  Unlimited 


333  Ravenswood  Avenue  -  Menlo  Park,  California  94025  -  (650)  859-4461 


Technical  Section 

Objective 

To  develop  a  de-centralized  system  for  allocating  mobile  sensor  assets  controlled  by  teams  of 
autonomous  air  vehicles  (A Vs)  and  deployed  in  spatially  complex  urban  environments.  The 
sensor  assets  are  to  support  AV  missions  involving  the  pursuit  of  one  or  more  mobile  intelligent 
ground  adversaries.  Since  sensor  information  is  typically  uncertain  and  incomplete,  AVs  must  be 
able  to  cooperate  to  coordinate  sensor  assignments  in  the  most  efficient  way  possible.  The 
environments  in  which  such  teams  are  deployed  are  spatially  complex,  introducing  additional 
uncertainty  arising  from  perceptual  occlusions  and  the  like.  In  addition  to  target  tracking,  we 
also  plan  to  develop  distributed  algorithms  for  dynamically  clustering  vehicles  around  objects  of 
interest  (e.g.,  to  protect  a  convoy).  Existing  approaches  to  coordinating  the  activities  of  AVs 
have  focused  on  essentially  2D  environments. 

(NB:  The  focus  of  this  effort  has  been  changed  somewhat  as  a  result  of guidance  from  the 
previous  ONR  Program  Manager,  Dr.  Allen  Moshfegh.  A  more  appropriate  title  for  this 
research  might  be:  Distributed  allocation  of  autonomous  air  vehicles  in  pursuit  of  intelligent 
adversaries  in  complex  urban  environments.) 

Approach 

Our  approach  is  divided  into  three  areas  of  work: 

1 .  Spatial  representation  and  reasoning,  including  path  planning.  Space  is  discretized 
into  a  distribution  of  AV  configurations  and  projections  of  adversaries  over  time  are 
computed.  We  make  use  of  3d  data  obtained  for  an  urban  environment  from  Planet  9 
Studios.  The  data  is  geo-referenced  using  SRI  geo- VRML. 

2.  Rich  simulation  environment  for  experimentation.  We  have  developed  an  OpenSim 
simulation  environment  in  which  alternative  initial  resource  and  target  arrangements  can 
be  defined  and  in  which  computed  flight  paths  can  be  displayed  during  resource  allocation. 

3.  Distributed  algorithm  design  and  experimentation  for  solving  the  AV  collaboration 
problem.  We  are  exploring  and  experimenting  with  several  different  approaches  to  this 
problem.  Our  first  made  use  of  an  auction  mechanism  for  task  allocation  and 
demonstrated  very  interesting  emergent  behavior  (discussed  in  more  detail  below). 
However,  we  found  that  a  purely  auction-based  system  for  allocating  the  best  resource  at 
each  round  may  lead  to  a  bad  solution:  for  instance,  the  choice  of  the  first  best  resource 
can  make  the  last  resource  impossible  to  allocate.  To  address  this  problem,  we  are 
developing  a  second  approach,  which  we  plan  to  experimentally  compare  with  the 
auctioning  approach.  The  second  approach  is  a  probabilistic  one,  that  handles  the 
potential  state  space  explosion  characteristic  of  conventional  MDP-based  approaches.  We 
apply  simulated  annealing  to  the  problem  of  minimizing  cost  and  maximizing  utility 
within  a  group  of  heterogeneous  resources. 
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Progress 

The  start  date  of  this  project  was  6  August  2003.  The  following  summarizes  our  progress 
relative  to  the  three  tasks  described  above. 

1 .  Spatial  representation  and  reasoning,  including  path  planning.  We  are  working  with 
models  of  cluttered  urban  environments  that  look  like  the  following  (showing  downtown 
San  Francisco). 


The  size  of  the  “world”  that  we  are  working  from  is  567  x  595  meters  (altitude  =  269  meters). 
The  space  is  discretized  in  a  regular  distribution  of  configurations  (roadmap).  A  configuration  is 
a  valid  position  in  the  space  for  a  UAV. 

We  generate  1237  collision-free  configurations  in  less  than  one  second  in  computing  a 
“roadmap”  of  navigatable  paths.  We  make  use  of  an  irregular  distribution  of  cubes  to  discretize 
the  space:  the  intuition  is  that  there  is  more  clutter  the  lower  one  is  to  the  ground.  This  step  is 
shown  in  the  figures  below. 
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Next,  we  compute  a  spanning  tree.  An  edge  is  added  between  two  configurations,  cl  and  c2,  in 
the  spanning  tree,  only  if  a  local  path  is  found  between  cl  and  c2  (using  a  local  path  planner) . 
This  results  in  one  component  with  approximately  12372  edges.  Computation  time  is 
approximately  20  seconds.  The  following  figure  shows  the  resulting  spanning  tree. 


The  final  step  involves  the  path  planning.  Path  planning  between  two  positions  PI  and  P2 
involving: 
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•  Connection  of  PI  and  P2  to  the  roadmap  (local  planning  giving  the  closest  reachable 
configurations  cl  and  c2) 

•  Path  planning  between  cl  and  c2  (A*)  in  the  roadmap 

•  Path  optimization  (smoothing) 

The  computation  time  per  path  is  a  few  seconds.  The  results  are  shown  in  the  following  figure. 


2.  Rich  simulation  environment  for  experimentation.  We  have  developed  a  simulation 
environment  for  testing  and  displaying  experiments  in  such  3D  environments.  Movies  of 
the  simulation  in  action,  showing  the  two  approaches  explored  so  far  and  discussed  in  (3) 
below,  are  available  at  our  project  website. 

3.  Distributed  algorithm  design  and  experimentation  for  solving  the  AV  collaboration 
problem.  In  the  auction-based  approach ,  a  set  of  valid  visibility  points  (VVP)  are 
computed.  Vehicle  Vi  reaching  a  valid  point  of  visibility  with  the  smallest  existing  time 
is  picked.  Vi  is  then  removed  from  the  vehicle  list.  Points  inside  the  “chasing  distance” 
are  removed  from  VVP.  A  coordinator/auctioneer  can  be  chosen  dynamically  and 
announces  the  set  of  future  time  points  that  need  to  be  covered.  From  one  simple 
negotiation  protocol,  3  unique  behaviors  (handoff,  tracking  and  helping)  emerge.  In  the 
probabilistic  approach  a  projection  module  first  generates  the  evolution  of  the 
environment  over  time.  Example:  for  the  tracking  task,  the  possible  moves  for  the  evader 
over  the  next  2  minutes.  For  a  convoy  protection  mission,  the  possible  moves  are  of  the 
convoy  and  the  enemies.  Time  is  then  discretized  into  quanta.  Each  configuration  is 
labeled  by  a  set  of  probabilities  Pi.  For  a  configuration  C,  Pi  gives  the  probability  for  a 
positive  event  (evader  presence,  convoy  presence)  at  t  =  i.  Now  that  each  configuration 
of  the  space  is  labeled  with  a  distribution  of  probabilities  over  the  time,  we  want  to  find 
the  best  resource  allocation: 

a.  to  maximize  the  positive  events  (Pi) 
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b.  to  minimize  a  Cost  function  (ex  :  fuel  used,  redundancy  with  the  other  allocations, 
shield  porosity  etc ...) 

c.  to  maximize  a  Utility  function  (ex  :  quality  of  observation,  communication  link  etc  ...). 

Each  task  has  a  specific  cost  and  utility  function.  This  optimization  is  done  over  n  quanta  of  time 
by  using  Simulated  Annealing.  Once  the  computation  is  done  (over  n  quanta),  the  allocations  for 
t  =  1  are  performed  by  the  UAVs.  The  environment  is  modified.  A  new  projection  is  computed 
and  a  new  allocation  is  computed  again. 

Cumulative  Statistics 


Journal  publications 

None. 

Conference  presentations/proceedings 

1 .  “Distributed  Robotic  Systems  in  Cluttered  Urban  Environments,  “  Symposium  on  Bridging 
the  Multiagent  and  Multirobotic  Research  Gap,  Stanford  University,  Spring  2004. 

2.  “Distributed  allocation  of  autonomous  air  vehicles  in  pursuit  of  intelligent  adversaries  in 
complex  urban  environments,”  (to  appear,  2004),  book  chapter,  Adversarial  Reasoning  edited 
by  Alex  Kott. 

3.  “Spatial  Reasoning  for  Very  Large  Scale  Robotic  Swarms  in  Urban  Settings,”  2nd  Annual 
Swarming:  Network  Enabled  C4ISR  Conference,  June  2004. 

Invention  disclosures/patents 

None. 

Awards  or  other  noteworthy  recognition 

None. 

Students/Postdocs  supported  by  this  grant 

Dr.  Benoit  Morisset  was  supported  as  a  postdoc. 
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Financial  Status 


Financial  Status  Report 

Inception  through  December  31,  2004 

Current  Period 

Cummulative 

Labor  Hrs.  Labor  $ 

Labor  Hrs. 

Labor  $ 

Labor 

124  $33,835 

3031 

$418,859 

Other  Direct  Costs 

Materials  and  Supplies 

$6,081 

Outside  Services 

$1,046 

Internal  Computing 

$2,472 

$41,506 

Shipping  &  Receiving 

$2,374 

Subscription  and  Books 

$8,430 

Meal  Expenses 

$1,005 

$6,392 

Travel  Expenses 

$4,039 

$42,702 

Telephone  Tolls 

$22 

$304 

Total  Other  Direct  Costs 

$7,538 

$108,835 

Total  Project  Expenses 

$41,373 

$527,694 

Project  Commitments 

Total  Expenses 

$527,694 
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